Preventive action based on estimated intent

ABSTRACT

An electronic device that selectively performs a preventive action is described. During operation, the electronic device may acquire, using one or more sensors, a measurement in an environment that is external to the electronic device, where the measurement provides information associated with an object, and the measurement is a non-contact measurement. Then, the electronic device may detect the object based at least in part on the measurement. Moreover, the electronic device may estimate an intent of the object based at least in part on the measurement. Next, when the estimated intent is associated with a type of adverse event, the electronic device may perform the preventive action prior to an occurrence of the type of adverse event, where the preventive action reduces a probability of the occurrence of the type of adverse event or an amount of financial damage associated with the occurrence of the type of adverse event.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a Continuation of U.S. Non-Provisional applicationSer. No. 16/599,121, “Preventive Action Based on Estimated Intent,”filed on Oct. 11, 2019, by Victor Shtrom, the contents of which areherein incorporated by reference.

BACKGROUND Field

The described embodiments relate to an electronic device that acquires anon-contact measurement in or of an environment that includes theelectronic device, and then selectively performs a predefined actionbased at least in part on the measurement.

Related Art

Security systems are often used to detect intrusion or unauthorizedentry into a region or an environment (e.g., a building). For example,when monitoring is activated, a burglar alarm may detect the presence ofa person (e.g., using a motion sensor) or a change in a state of aportal (such as a door or a window). In response, the burglar alarm maysound in an attempt to deter the intruder.

However, these capabilities are typically based on predefined or fixedlogic. Thus, an alarm may sound when any person is in proximity orwhenever the state of the portal changes. Paradoxically, this can reducethe effectiveness of a security system. For example, numerous falsealarms can lead to complacency, so that subsequent alarms are ignored,and are often irritating.

SUMMARY

In a first group of embodiments, an electronic device that selectivelyperforms a preventive action is described. This electronic device mayinclude: one or more sensors; and an integrated circuit. Duringoperation, the integrated circuit acquires, using the one or moresensors, a measurement in an environment that is external to theelectronic device, where the measurement provides information associatedwith an object, and the measurement is a non-contact measurement. Then,the integrated circuit detects the object based at least in part on themeasurement. Moreover, the integrated circuit estimates an intent of theobject based at least in part on the measurement. Next, when theestimated intent is associated with a type of adverse event, theintegrated circuit performs the preventive action prior to an occurrenceof the type of adverse event, where the preventive action reduces aprobability of the occurrence of the type of adverse event or an amountof financial damage associated with the occurrence of the type ofadverse event.

For example, the one or more sensors may include a transmitter thattransmits wireless signals and a receiver that receives wireless-returnsignals. In some embodiments, the one or more sensors include two ormore different types of sensors and the measurement is acquired usingthe two or more different types of sensors.

Moreover, the object may include a person, the integrated circuit mayidentify the person, and the intent may be estimated based at least inpart on the identity of the person. Alternatively, the object mayinclude a person having an unknown identity, and the intent may beestimated based at least in part on an association of the person withone or more prior occurrences of the type of adverse event in an eventhistory.

Furthermore, the type of adverse event may include: theft, propertydamage, and/or another type of crime.

Additionally, the measurement may be acquired using a first sensor inthe one or more sensors, and the preventive action may includeacquiring, using a second sensor in the one or more sensors, a secondmeasurement that provides information associated with the object.

In some embodiments, the electronic device may include a light source,and the preventive action may include selectively illuminating theobject using the light source. Alternatively or additionally, theelectronic device may include an alarm, and the preventive action mayinclude selectively activating the alarm. Furthermore, the electronicdevice may include a display, and the preventive action may includeselectively presenting information about the object on the display.

Note that the preventive action may include providing information aboutthe object addressed to a law enforcement agency and/or contacting a lawenforcement agency.

Moreover, the electronic device may include a lock, and the preventiveaction may include: determining a state of the lock; providing anelectronic signal that sets the lock into a locked state when the lockis initially in an unlocked stated; and disabling an ability to changethe state of the lock. Alternatively or additionally, the electronicdevice may include a vehicle, and the preventive action may includedisabling movement of the vehicle.

Furthermore, the electronic device may include a battery, and theintegrated circuit may perform at least one of the acquiring, thedetecting, the estimating and the performing using a dynamic subset ofresources in the electronic device based at least in part on: adischarge current of the battery and/or a remaining charge of thebattery.

Additionally, the object may include a person, and the intent may beestimated based at least in part on an inferred emotional state of theperson and/or a behavior of the person.

In some embodiments, the measurement may provide information associatedwith a second object that is related to the object, the integratedcircuit may identify the second object, and the intent may be estimatedbased at least in part on the identified second object.

Moreover, the object may include a person, and the estimated intent maybe based at least in part on a vital sign of the person, which isspecified by the measurement.

Another embodiment provides a vehicle that includes the electronicdevice.

Another embodiment provides a computer-readable storage medium for usewith the electronic device. This computer-readable storage medium mayinclude program instructions that, when executed by the electronicdevice, causes the electronic device to perform at least some of theaforementioned operations of the electronic device.

Another embodiment provides a method. This method includes at least someof the operations performed by the electronic device.

This Summary is provided for purposes of illustrating some exemplaryembodiments, so as to provide a basic understanding of some aspects ofthe subject matter described herein. Accordingly, it will be appreciatedthat the above-described features are examples and should not beconstrued to narrow the scope or spirit of the subject matter describedherein in any way. Other features, aspects, and advantages of thesubject matter described herein will become apparent from the followingDetailed Description, Figures, and Claims.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a drawing illustrating an example of an environment thatincludes a vehicle in accordance with an embodiment of the presentdisclosure.

FIG. 2 is a block diagram illustrating an example of an electronicdevice that can included on the vehicle in FIG. 1 in accordance with anembodiment of the present disclosure.

FIG. 3 is a block diagram illustrating an example of a data structurefor use in conjunction with, e.g., the electronic device of FIGS. 1 and2 in accordance with an embodiment of the present disclosure.

FIG. 4 is a flow diagram illustrating an example of a method forselectively performing a preventive action in accordance with anembodiment of the present disclosure.

FIG. 5 is a drawing illustrating an example of communication amongcomponents, e.g., in the electronic device of FIGS. 1 and 2 inaccordance with an embodiment of the present disclosure.

FIG. 6 is a drawing illustrating an example of selective performing of apreventive action using, e.g., the electronic device of FIGS. 1 and 2 inaccordance with an embodiment of the present disclosure.

FIG. 7 is a drawing illustrating an example of an environment within avehicle in accordance with an embodiment of the present disclosure.

FIG. 8 is a flow diagram illustrating an example of a method forselectively providing a recommendation in accordance with an embodimentof the present disclosure.

FIG. 9 is a drawing illustrating an example of communication amongcomponents, e.g., in the electronic device of FIGS. 1 and 2 inaccordance with an embodiment of the present disclosure.

FIG. 10 is a drawing illustrating an example of selective providing arecommendation using, e.g., the electronic device of FIGS. 1 and 2 inaccordance with an embodiment of the present disclosure.

FIG. 11 is a drawing illustrating an example of an environment within avehicle in accordance with an embodiment of the present disclosure.

FIG. 12 is a flow diagram illustrating an example of a method forautomatically modifying an environmental control in accordance with anembodiment of the present disclosure.

FIG. 13 is a drawing illustrating an example of communication amongcomponents, e.g., in the electronic device of FIGS. 1 and 2 inaccordance with an embodiment of the present disclosure.

FIG. 14 is a drawing illustrating an example of automatically modifyingan environmental control using, e.g., the electronic device of FIGS. 1and 2 in accordance with an embodiment of the present disclosure.

FIG. 15 is a block diagram illustrating an example of an electronicdevice in accordance with an embodiment of the present disclosure.

Note that like reference numerals refer to corresponding partsthroughout the drawings. Moreover, multiple instances of the same partare designated by a common prefix separated from an instance number by adash.

DETAILED DESCRIPTION

In a first group of embodiments, an electronic device that selectivelyperforms a preventive action is described. During operation, theelectronic device may acquire, using one or more sensors, a measurement(such as a radar measurement) in an environment that is external to theelectronic device, where the measurement provides information associatedwith an object, and the measurement is a non-contact measurement. Then,the electronic device may detect the object based at least in part onthe measurement. Moreover, the electronic device may estimate an intentof the object based at least in part on the measurement. For example,the object may include a person, and the intent may be estimated basedat least in part on an inferred emotional state of the person and/or abehavior of the person. Next, when the estimated intent is associatedwith a type of adverse event (such as a type of crime), the electronicdevice may perform the preventive action prior to an occurrence of thetype of adverse event, where the preventive action reduces a probabilityof the occurrence of the type of adverse event or an amount of financialdamage associated with the occurrence of the type of adverse event.

By selectively performing the preventive action, these securitytechniques may prevent or mitigate the type of adverse event. Moreover,by doing so in an intelligent manner (e.g., based at least in part onthe estimated intent, the security techniques may reduce or eliminatefalse alarms. Consequently, the security techniques may be moreeffective, because alerts (such as an alarm) that is selectively soundedmay not be ignored. In addition, the security techniques may be lessannoying and, thus, may improve the user experience, while moreeffectively providing protection.

In the discussion that follows, radar is used as an illustrative exampleof a measurement or sensor technique. For example, the radar may involveradar signals having a fundamental frequency of 1-10 GHz, 24 GHz, 77-81GHz, 140 GHz, and/or another electromagnetic signal having a fundamentalfrequency in the radio or microwave frequency band. Moreover, the radarsignals may be continuous wave and/or pulsed, may modulated (such asusing frequency modulation or pulse modulation) and/or may be polarized(such as horizontal polarization, vertical polarization or circularpolarization). Notably, the radar signals may be frequency-modulatedcontinuous-wave, pulse-modulated continuous-wave, multiple-inputmultiple-output (MIMO), etc. However, a wide variety of measurement orsensor techniques may be used in conjunction with or to implement thedisclosed embodiments. For example, the measurement or sensor techniquesmay include: optical imaging in the visible spectrum or a visiblefrequency band, infrared, sonar, FLIR, optical imaging having a dynamicrange or contrast ratio exceeding a threshold value (such as 120 dB),lidar, an acoustic measurement (e.g., using an acoustic signal in anaudible frequency band), an ultrasound measurement, etc. While many ofthe embodiments use one or more non-contact measurement techniques, inother embodiments one or more ‘direct’ (in-contact) measurementtechniques may be used, such as: a vital sign measurement, an impedancemeasurement (such as an galvanometric measurement of skin impedance, oran AC or DC impedance measurement), etc.

Moreover, in the discussion that follows, the electronic device maycommunicate using one or more of a wide variety of communicationprotocols. For example, the communication may involve wired and/orwireless communication. Consequently, the communication protocols mayinclude: an Institute of Electrical and Electronics Engineers (IEEE)802.11 standard (which is sometimes referred to as ‘Wi-Fi®’ from theWi-Fi Alliance of Austin, Tex.), Bluetooth® (from the Bluetooth SpecialInterest Group of Kirkland, Wash.), another type of wireless interface(such as another wireless-local-area-network interface), acellular-telephone communication protocol (e.g., a 3G/4G/5Gcommunication protocol, such as UMTS, LTE), an IEEE 802.3 standard(which is sometimes referred to as ‘Ethernet’), etc. In the discussionthat follows, Ethernet and universal serial bus (USB) are used asillustrative examples.

We now describe some embodiments of the security and the feedbacktechniques. FIG. 1 presents a drawing illustrating an example of anenvironment 100 that includes a vehicle 110. For example, vehicle 110may include: a car or automobile, a bus, a truck, etc., and moregenerally one that includes one or more non-retractable wheels incontact with a surface (such as a road or the ground) during operation.As described further below with reference to FIG. 2, vehicle 110 mayinclude an electronic device (E.D.) 112 that collects or acquires one ormore measurements associated with different types of sensors. Notably,while stationary (e.g., when parked), vehicle 110 may acquirement one ormore measurements using a sensor (51) 114. As discussed further below,in some embodiments vehicle 110 may optionally acquire (either with theone or more measurements or subsequently as part of a predefinedpreventive action) one or more additional measurements using optionalsensor (S2) 116, where sensor 114 is a different type of sensor thansensor 116. Note that sensors 114 and 116 may be included in electronicdevice 112. Alternatively, sensors 114 and 116 may be coupled toelectronic device 112, such as by electrical signal lines, opticalsignal lines, a cable and/or a bus, which is illustrated by signal lines108.

Moreover, in order to obtain accurate and useful sensor informationabout environment 100, sensors 114 and 116 may be in the same plane ormay be coplanar in plane 126. In addition, apertures 118 of orassociated with sensors 114 and 116 may be adjacent to each other or maybe co-located (i.e., at the same location on vehicle 110). This mayensure that sensors 114 and 116 capture or obtain sensor information ofsubstantially the same portions of environment 100 and objects (such asobject 120) in environment 100. Therefore, sensors 114 and 116 may haveat least substantially overlapping fields of view in environment 100,such as fields of view that are more than 50, 75, 80 or 90% in common.

In some embodiments, sensor 114 performs radar measurements of radarinformation, and sensor 116 performs optical imaging in a visiblespectrum or a visible frequency band (such as at least a frequencybetween 430 and 770 THz or at least a wavelength between 390 and 700nm). However, more generally, sensors 114 and 116 may perform at least apair of different measurements. For example, sensors 114 and 116 mayinclude two or more of: a radar sensor, an optical imaging sensor in thevisible spectrum or the visible frequency band, an infrared sensor, aFLIR sensor, a sonar sensor, an optical imaging sensor having a dynamicrange or contrast ratio exceeding a threshold value (such as 120 dB),lidar, etc. More generally, as described further below with reference toFIGS. 3-6, sensor 114 may acquire one or more measurements of object 120(such as a person) that, using one or more estimation techniques (suchas a pre-trained or a dynamically adapted/self-learning predictivemodel, e.g., a machine-learning classifier or regression model, and/or aneural network), may be estimated to have an intent. When the estimatedintent is associated with a type of adverse event (such as theft,property damage, and/or another type of crime), electronic device 112and/or vehicle 110 may perform a preventive action, prior to anoccurrence of the type of adverse event, in order to prevent theoccurrence of the type of event, reduce a probability of the occurrenceof the type of adverse event, or reduce an amount of financial damageassociated with the occurrence of the type of adverse event. Note that agiven sensor may be capable of transmitting and/or receiving signals.

FIG. 2 presents a block diagram illustrating an example of electronicdevice 112. As described further below with reference to FIG. 15, thiselectronic device may include a control engine 210 (such as anintegrated circuit and/or a processor, which are sometimes referred toas ‘control logic’) that performs at least some of the operations in thesecurity or feedback techniques. Notably, control engine 210 mayprovide, via interface circuit 208, one or more signals or instructionsto sensor 114 and optional sensor 116 (which may be included in orcoupled to electronic device 112, and thus are optional in electronicdevice 112 in FIG. 2) to acquire, respectively, one or more measurementsin an environment that includes electronic device 112. For example,sensor 114 may measure radar information, and sensor 116 may measure anoptical image. Then, sensor 114 and/or sensor 116 may provide the one ormore measurements to control engine 210. In some embodiments, the one ormore measurements include one or more associated timestamps when the oneor more measurements were acquired. Alternatively, control engine 210may generate the one or more timestamps when the one or moremeasurements are received.

Next, control engine 210 may store the one or more measurements (orinformation that specifies one or more results of the one or moremeasurements) in memory 212. For example, as described further belowwith reference to FIG. 3, the one or more measurements may be stored inassociated memory locations. In some embodiments, the one or moremeasurements are stored in memory along with the one or more timestampsand/or location information that specifies where the one or moremeasurements were acquired (such as GPS information, locationinformation associated with a local positioning system, e.g., from aWLAN, location information associated with a cellular-telephone network,etc.).

Furthermore, control engine 210 may optionally perform one or morequality-control operations on the one or more measurements. For example,control engine 210 may analyze a light intensity or luminance level inan optical image and may compare the luminance level to a thresholdvalue. Alternatively or additionally, control engine 210 may analyze theoptical image or the radar information to determine a signal-to-noiseratio, and then may compare the signal-to-noise ratio to anotherthreshold value. In some embodiments, control engine 210 may analyze theone or more measurements to confirm that each include informationassociated with the same object (such as object 120 in FIG. 1).

Based on the results of the one or more quality-control operations,control engine 210 may perform a remedial action. For example, controlengine 210 may store a quality-control metric with the one or moremeasurements in memory 212, such as a quality-control metric thatindicates ‘pass’ (such as when the luminance level exceeds the thresholdvalue, the signal-to-noise ratio exceeds the other threshold valueand/or that the one or more measurements include information associatedwith the same object), ‘fail’ (such as when the luminance level is lessthan the threshold value, the signal-to-noise ratio is less than theother threshold value and/or the one or more measurements do not includeinformation associated with the same object) or ‘further analysisrequired’ (such as when the results of the one or more quality-controloperations are mixed). Alternatively, control engine 210 may erase theone or more measurements when either fails the one or morequality-control operations.

Separately or additionally, in some embodiments quality control isoptionally performed while the one or more measurements are acquired.For example, when performing a measurement using sensor 114 and/or ameasurement using sensor 116, control engine 210 (and/or sensor 114 orsensor 116, respectively) may determine an environmental condition (suchas light intensity, e.g., a luminance level, a weather condition such asfog, a temperature, e.g., greater than 90 F, etc.) and/or informationassociated with an object (such as object 120 in FIG. 1). For example,control engine 210 may determine whether the object is two dimensional(such as a sign) or three dimensional (such as a person or an animal).Then, based on the determined environmental condition and/or informationassociated with the object, control engine 210 (and/or sensor 114 orsensor 116, respectively) may perform a remedial action. For example,control engine 210 may increase a transmit power of radar signals, ormay use different filtering and/or a longer integration time inconjunction with an infrared sensor. Alternatively or additionally,control engine 210 may provide one or more signals or instructions to alight source in or coupled to electronic device 112 (such as a lightsource included in or associated with sensor 114 and/or 116) so thatselective illumination is output, such as a two or three-dimensionalarray of dots, a pattern of stripes, an illumination pattern, etc. Thisselective illumination may improve the ability to resolve structure of athree-dimensional object. In some embodiments, control engine 210 mayprovide one or more signals or instructions to a light source in orcoupled to electronic device 112 (such as a light source, e.g., avertical-cavity surface-emitting laser or vcsel, included in orassociated with sensor 114 and/or 116) so that illumination having awavelength is output (e.g., illumination at a particular wavelength).This constant-wavelength illumination may allow the one or moremeasurements to be acquired when the signal-to-noise ratio is low.

Note that electronic device 112 may be positioned on or proximate to asurface of vehicle 110 in FIG. 1 (such as a front, back or sidesurface). Alternatively, electronic device 112 may be mounted orpositioned on a top surface (such as a roof) of vehicle 110 and anaperture of electronic device 112 may rotate about a vertical axis, sothat it ‘sweeps’ an arc (e.g., 120°, 180° or 360°).

In some embodiments, electronic device 112 (FIGS. 1 and 2) and/orvehicle 110 includes fewer or additional components, two or morecomponents are combined into a single component and/or positions of oneor more components are changed.

FIG. 3 presents a block diagram illustrating an example of a datastructure 300 for use in conjunction with electronic device 112 (FIGS. 1and 2), such as in memory in or associated with electronic device 112(FIGS. 1 and 2). Notably, data structure 300 may include: one or moreinstances 308 of measurement 310, optional associated timestamps 312 formeasurements 310, one or more instances of additional measurements 314,optional associated timestamps 316 for additional measurements 314, oneor more quality-control (Q.-C.) metrics 318 associated with the one ormore instances of measurements 310 and/or associated with the one ormore instances of measurements 314, and/or location information 320where the one or more instances of measurements 310 and/or 314 wereacquired. Note that data structure 300 may include fewer or additionalfields, two or more fields may be combined, and/or a position of a givenfield may be changed. For example, data structure may includeinformation extracted from measurements 310 and/or additionalmeasurements 314. In some embodiments, where measurements 310 includeradar measurements, the extracted information may include a range to theobject, an angle of arrival of the radar measurements and/or a velocityof the object (such as based at least in part on Doppler measurements).

FIG. 4 presents a flow diagram illustrating an example of a method 400for selectively performing a preventive action. This method may beperformed by an electronic device (such as electronic device 112 inFIGS. 1 and 2) or a component in the electronic device (such as anintegrated circuit or a processor). During operation, the electronicdevice may acquire, using one or more sensors, a measurement (operation410) in an environment that is external to the electronic device, wherethe measurement provides information associated with an object, and themeasurement is a non-contact measurement. For example, the one or moresensors may include a transmitter that transmits wireless signals and areceiver that receives wireless-return signals. In some embodiments, theone or more sensors include two or more different types of sensors (suchas a radar sensor and an optical sensor), and the measurement isacquired using the two or more different types of sensors.

Then, the electronic device may detect the object (operation 412) basedat least in part on the measurement. For example, the electronic devicemay apply one or more pretrained neural networks (e.g., a convolutionalneural network) to the measurement to detect and/or to identify/classifythe object. The one or more neural networks may be arranged in or maydefine a classification hierarchy to iteratively detect and/or identifythe object, such as animal/non-animal, then human/non-human animal,etc., vehicle/non-vehicle, type of vehicle, etc., or streetsign/non-street sign, type of street sign, etc. Alternatively oradditionally, a wide variety of analysis and/or identificationtechniques may be used to extract features from the measurement (e.g.,an image), such as one or more of: normalizing a magnification or a sizeof the object, rotating the object to a predefined orientation,extracting the features that may be used to detect and/or identify theobject, etc. Note that the extracted features may include: edgesassociated with one or more potential objects, corners associated withthe potential objects, lines associated with the potential objects,conic shapes associated with the potential objects, color regions in themeasurement, and/or texture associated with the potential objects. Insome embodiments, the features are extracted using a descriptiontechnique, such as: scale invariant feature transform (SIFT), speed-uprobust features (SURF), a binary descriptor (such as ORB), binary robustinvariant scalable keypoints (BRISK), fast retinal keypoint (FREAK),etc. Furthermore, the electronic device may apply one or more supervisedor machine-learning techniques to the extracted features to detectand/or identify/classify the object, such as: support vector machines,classification and regression trees, logistic regression, LASSO, linearregression and/or another (linear or nonlinear) supervised-learningtechnique.

Moreover, the electronic device may extract a signature associated withthe object from the measurement (e.g., radar information). Extractingthe signature may involve at least some of the processing of reflectedradar signals to extract radar information. For example, the electronicdevice may perform windowing or filtering, one or more Fourier ordiscrete Fourier transforms (with at least 128 or 256 bits), peakdetection, etc. In some embodiments, a constant false alarm rate (CFAR)technique is used to detect and/or determine whether a peak in the radarinformation is significant. Notably, the electronic device may calculatestatistical metrics (such as a mean and a standard deviation) for agiven range, and the electronic device may determine if a given peak issignificant based on the calculated statistical metrics at differentranges. This approach may allow the electronic device, separately or inconjunction with processing by a remotely located electronic device(such as a cloud-based computer), to statistically detect and/oridentify the radar information of the object.

The resulting signature of the object may include multiple dimensions.For example, the signature may include one or more of: a range to theobject, a first angle to the object along a first axis (such as ahorizontal axis), Doppler information associated with the object and/ora second angle to the object along a second axis (such as a verticalaxis).

Moreover, the electronic device may estimate an intent of the object(operation 414) based at least in part on the measurement (and/orinformation extracted from the measurement or that is determined basedat least in part on analysis of the measurement). For example, theobject may include or may be a person, the electronic device mayidentify the person (such as by comparing the measurement to apredetermined dataset of measurements and associated individualidentities), and the intent may be estimated based at least in part onthe identity of the person. Notably, if the person is known and isconsidered ‘friendly’ (such as a registered owner of the electronicdevice, or a person that has been previously identified one or moretimes and who is consequently considered low risk or harmless), theestimated intent may be ‘neural’ or ‘positive’ (and, thus, a risk for atype of adverse event, such as theft, property damage, and/or anothertype of crime, may be low). Alternatively, the object may include or maybe a person having an unknown identity, and the intent may be estimatedbased at least in part on an association of the person with one or moreprior occurrences of the type of adverse event in an event history. Forexample, while the person may be unknown, there may be one or morerepeated occurrences of the type of adverse event when the unknownperson was in proximity (such as reported or captured occurrences thatare stored, locally or remotely, in an event history). Consequently, inthis example, the unknown person may be deemed ‘unfriendly’, so thecorresponding estimated intent may be ‘negative’ (and, thus, a risk forthe type of adverse event may be high).

Note that in embodiments where the object may include or may be aperson, the intent may be estimated based at least in part on: aninferred emotional state of the person (such as a facial expression, aposture, a tone of voice, etc., that indicates that the person is angryor hostile); a behavior of the person (such as when the person appearsintoxicated or under the influence of a drug, when the person appearsviolent, when the person is acting secretive or in a surreptitiousmanner, e.g., the person is acting like they are doing something wrongor that they have something to hide, when the person's face is at leastpartially hidden or obscured, etc.); and/or a vital sign of the person(such as a pulse rate or a respiration rate), which is specified by themeasurement (such as a radar measurement). For example, an emotionalstate may be estimated or inferred by using one or more measurements(such as the one or more measurements) as inputs to a pretrained neuralnetwork or a machine-learning model that outputs an estimate of anemotional state (such as a probability of a particular emotional stateor a classification in a particular emotional state). In someembodiments, the measurement may provide information associated with asecond object that is related to the object, the electronic device mayidentify the second object (such as a tool carried by the person, e.g.,a crowbar, spray paint or a brick, which may be identified by comparingthe measurement to a predetermined annotated dataset of measurements andassociated classifications), and the intent may be estimated based atleast in part on the identified second object.

Next, when the estimated intent is associated with the type of adverseevent (operation 416), the electronic device may perform the preventiveaction (operation 418) prior to an occurrence of the type of adverseevent, where the preventive action reduces a probability of theoccurrence of the type of adverse event or an amount of financial damageassociated with the occurrence of the type of adverse event. Otherwise(operation 416), the electronic device may repeat method 400, e.g., atoperation 410.

For example, the measurement may be acquired using a first sensor in theone or more sensors, and the preventive action may include acquiring,using a second sensor in the one or more sensors, a second measurementthat provides information associated with the object. Notably, themeasurement may include a radar measurement, and the second measurementmay include an image (such as a picture or a video), which may be storedin memory 212 (FIG. 2), in remotely located memory (such as incloud-based storage), presented on a display in a vehicle, and/orprovided to a remotely located electronic device or computer (such asanother electronic device of or associated with: an owner of theelectronic device or the vehicle, a security service, and/or lawenforcement). More generally, the preventive action may includeproviding information about the object to a law enforcement agencyand/or contacting a law enforcement agency (e.g., an indicating that atype of adverse event may be occurring or may be about to occur).

In some embodiments, the electronic device may include a light source,and the preventive action may include selectively illuminating theobject using the light source. Alternatively or additionally, theelectronic device may include an alarm, and the preventive action mayinclude selectively activating the alarm. Furthermore, the electronicdevice may include a display, and the preventive action may includeselectively presenting information about the object on the display(e.g., as noted previously, the measurement, such as an image of theobject, may be presented on the display).

Moreover, the electronic device may include a lock, and the preventiveaction may include: determining a state of the lock; providing anelectronic signal that sets the lock into a locked state when the lockis initially in an unlocked stated; and disabling an ability to changethe state of the lock. Alternatively or additionally, the electronicdevice may include a vehicle, and the preventive action may includedisabling movement of the vehicle.

In some embodiments, the electronic device may perform one or moreoptional additional operations (operation 420). For example, theelectronic device may include a battery, and the electronic device mayperform at least one of the acquiring (operation 410), the detecting(operation 412), the estimating (operation 414) and the performing(operation 418) using a dynamic subset of resources in the electronicdevice based at least in part on: a discharge current of the batteryand/or a remaining charge of the battery.

Note that the electronic device may be a portable or removableelectronic device, such as a measurement or sensor module that isinstalled in or integrated into the vehicle.

In embodiments in which the one or more measurements include radarmeasurements, the electronic device may measure radar information usinga variety of antenna configurations. For example, the electronic device112 (or sensor 114 in FIGS. 1 and 2) may include multiple antennas.Different subsets of these antennas may be used to transmit and/orreceive radar signals. In some embodiments, a first subset of theantennas used to transmit radar signals and a second subset of theantennas used to receive reflected radar signals may be dynamicallyadapted, such as based on environmental conditions, a size of theobject, a distance or range to the object, a location of the objectrelative to a vehicle, etc. By varying the number of transmit andreceive antennas, the angular resolution of the radar can be changed.Notably, one transmit antenna and four receive antennas (1T/4R) may havean angular resolution of approximately 28.5°, two transmit antennas andfour receive antennas (2T/4R) may have an angular resolution ofapproximately 15°, three transmit antennas and four receive antennas(3T/4R) may have an angular resolution of approximately 10°, etc. Moregenerally, the angular resolution with one transmit antenna and Nreceive antennas (1T/NR), where N is a non-zero integer, may beapproximately 114°/N. Thus, if there is one transmit antenna and 192receive antennas, the angular resolution may be less than one degree.For automotive applications, an angular resolution may be a few degreesat a distance of 200-250 m. As described further below with reference toFIG. 15, note that the transmit and/or receive antennas may be physicalantennas or may be virtual antennas in an adaptive array, such as aphased array.

Moreover, in some embodiments, the transmit antenna(s) has 6-30 dB gain,a beam width between a few degrees and 180°, a transmit power of up to12 dBm, and an effective range of up to 200-250 m. Furthermore, theremay be one transmit antenna and one receive antenna (1T/1R), threetransmit antennas and four receive antennas (1T/4R), three transmitantenna and four receive antennas (3T/4R), MIMO for spatial diversity,etc. Furthermore, the location(s) or positions of the transmit and/orthe receive antenna(s) may be selected to increase a horizontal and/or avertical sensitivity. For example, an antenna may be displaced relativeto another antenna along a vertical or a horizontal axis or direction byone half of a fundamental or carrier wavelength of the radar signals toincrease the (respectively) vertical or horizontal sensitivity.

Embodiments of the security techniques are further illustrated in FIG.5, which presents a drawing illustrating an example of communicationamong components in electronic device 112, vehicle 110 and electronicdevice 128. Notably, sensor(s) 510 in electronic device 112 may acquireone or more measurements 512 in an environment that is external toelectronic device 112, where the one or more measurements 512 provideinformation 514 associated with an object 516, and the one or moremeasurements 512 include a non-contact measurement. Then, an integratedcircuit (I.C.) 518 in electronic device 112 may detect object 516 basedat least in part on the one or more measurements 512. For example,integrated circuit 518 may analyze the one or more measurements 512 todetermine information 514, which is then used to detect object 516. Insome embodiments, integrated circuit 518 may optionally identify 520object 516.

Moreover, integrated circuit 518 may estimate an intent 522 of object516 based at least in part on the one or more measurements 512,information 514, and/or identification 520. When the estimated intent522 is associated with a type of adverse event 524, integrated circuit518 may perform a preventive action 526 prior to an occurrence of thetype of adverse event 524.

For example, integrated circuit 518 may provide instruction 528 tocommunication circuit (C.C.) 530 in electronic device 112, whichprovides instruction 528 to vehicle 110 (e.g., in one or more packets orframes). In response to instruction 528, vehicle 110 may: turn on alight source, activate an alarm, change a state of a lock, disablemovement of vehicle 110, present information about object 516 on adisplay in vehicle 110 (such as the one or more measurements 512,information 514 and/or identity 520), etc. Alternatively oradditionally, integrated circuit 518 may provide instruction 528 tocommunication circuit 530, which then provides a message 532 toelectronic device 128 (which may be owned by or associated with an owneror user of electronic device 112, a security service, or a lawenforcement agency). This message may include information about object516, such as the one or more measurements 512, information 514, identity520, and/or type of adverse event 524. Note that this information may beaggregated into an event history, which may be subsequently accessed byvehicle 110 and/or another vehicle in order to assist in estimatingintent 522 during one or more future events. Alternatively oradditionally, information 534 corresponding to the event history may bestored in memory 536 in electronic device 112.

While the preceding discussion illustrated electronic device 112 locallyperforming operations in the security techniques (e.g., in real-time),in other embodiments at least some of the operations may be, at least inpart, performed by another electronic device (such as vehicle 110, i.e.,in proximity to electronic device 112, and/or a remotely locatedelectronic device, such as a cloud-based computer 122 in FIG. 1). Forexample, electronic device 112 may pre-process the one or moremeasurements 512, and detection of object 516, determining of identity520 and/or estimating intent 522 may be performed, at least in part, bya cloud-based computer 122 (FIG. 1) that is accessed via communicationusing network 124 in FIG. 1 (such as via wireless communication in aWLAN or a cellular-telephone network and/or via wireless communication,e.g., using the Internet).

FIG. 6 presents a drawing illustrating an example of selectiveperforming of a preventive action using electronic device 112 (FIGS. 1and 2). Notably, sensor(s) in electronic device 112 may, continuously,periodically or as needed, acquire one or more measurements in anenvironment that is external to electronic device 112, where the one ormore measurements provide information associated with object 120, andthe one or more measurements include a non-contact measurement. Forexample, the one or more measurements may include radar measurements 610and an optical measurement (such as capturing an image 612). Then,electronic device 112 may analyze the radar measurements 610 to detectobject 120, including extracting a signature 614 associated with object120 and/or to determine one or more vital signs (such as a pulse and/ora respiration rate). Moreover, electronic device 112 may analyze image612 to detect object 120. In some embodiments, electronic device 112 mayuse information in image 612 to identify object 120 (such as inputs to apretrained neural network or a machine-learning model). This may includeclassifying object 120 and/or identifying a particular person (such asby using face or biometric recognition). In general, the informationprovided by the one or more measurements may be compared or jointlyanalyzed in order to detect and/or to identify object 120.

Next, electronic device 112 may estimate an intent of object 120. Forexample, based at least in part on signature 614 and/or the identity ofobject 120, electronic device 112 may determine whether object 120 (suchas a particular person) is known and considered friendly or safe, ornot. This may involve a comparison with historical records of previousmeasurements on objects and subsequent events (such as the occurrence orthe absence of an occurrence of a type of adverse event) by electronicdevice 112 and/or one or more other instances of electronic device 112(which may be shared or may be stored in memory in a remotely accessiblecloud-based computer), predefined relationships with electronic device112 (such as a specified owner of electronic device 112 or a trustedperson), an inferred relationship (such as an individual who has beenpreviously observed to be in proximity to the specified owner ofelectronic device 112 or a trusted person), etc. Note that thehistorical records of previous measurements on objects and subsequentevents may be used even if the identity of a person is unknown. Forexample, if an unknown person was involved in car theft in the area,electronic device 112 may compare signature 614 and the image with therecords to confirm that it is the same unknown person, and then may usethe prior association with a type of event to ‘post-did’ (using priorbehavior, instead of predict) that this unknown person's estimate intentis negative or that there is a high risk for the type of adverse event.Alternatively or additionally, electronic device 112 may access policereports and/or publicly available criminal records to estimate theintent of a person.

Note that the intent may be estimated based at least in part on aninferred emotional state of the person and/or a behavior of the person.For example, the one or more measurements may indicate that a person isangry and/or drunk. In these circumstances, electronic device 112 mayestimate the intent of the person as negative or hostile, and thus thatthere is a high risk for a type of adverse event. Alternatively oradditionally, if the person is carrying a second object (such as acrowbar, burglary tools, a weapon, or spray paint), electronic device112 may estimate the intent of the person as negative or hostile, andthus, once again, that there is a high risk for a type of adverse event.For example, the radar measurements may detect and/or identify (such asclassify) the second object, even when it is under a person's clothingor in a backpack. In some embodiments, the intent is estimated based onthe vital sign. For example, an elevated vital sign may indicate fear orsuspicious activity. In conjunction with the image, electronic device112 may estimate the emotional state of a person.

As discussed previously, when the estimated intent is associated with atype of adverse event (or an increased risk for the type of adverseevent), electronic device 112 may perform a preventive action prior toan occurrence of the type of adverse event.

While FIGS. 2-6 illustrated the use of electronic device 112 to provideenhanced security by monitoring potential threats in an externalenvironment (such as outside of a vehicle), in other embodimentselectronic device 112 may be used within a vehicle to improveself-driving technology. FIG. 7 presents a drawing illustrating anexample of an environment within a vehicle 710. In this environment, aperson 712 may be sitting in a driver's seat or a driver's position 714in vehicle 710. As discussed further below, in principle, even when apartial or fully autonomous vehicle application is operating vehicle710, person 712 may need to take over operation of vehicle 710 on shortnotice. However, if person 712 is distracted or unaware about what iscurrently happening around vehicle 710 (and what is potentially about tohappen), transferring control to person 712 may be ill-advised, becausean accident may still occur or an even worse outcome may result. Thechallenge, in this regard, is to be able to determine when person 712 isproperly prepared to assume control. This is different that simplydetecting whether person 712 has their hands on a steering wheel or hastheir foot on an accelerator or a break petal. Embodiments of thefeedback techniques may be used to address this problem.

This is shown in FIG. 8, which presents a flow diagram illustrating anexample of a method 800 for selectively providing a recommendation. Thismethod may be performed by an electronic device (such as electronicdevice 112 in FIGS. 1 and 2) or a component in the electronic device(such as an integrated circuit or a processor). During operation, theelectronic device may acquire, using one or more sensors, a measurement(operation 810) within an environment that is external to the electronicdevice, where the measurement provides information associated with aperson located in a driver's position in a vehicle, and the measurementincludes a non-contact measurement. For example, the one or more sensorsmay include a transmitter that transmits wireless signals and a receiverthat receives wireless-return signals. Alternatively or additionally,the one or more sensors may include an image sensor that performsoptical measurements (such as capturing one or more images or a video).

Then, the electronic device may assess situational awareness (operation812) of the person based at least in part on the measurement. Note that‘situational awareness’ may include perception of environmental elementsand events with respect to time or space, the comprehension of theirmeaning, and/or the projection of their future status. When the personor potential driver has appropriate or sufficient situational awareness,they may be better able to safely and effectively assume control of thevehicle when needed. Thus, the situational awareness may indicate anawareness of the person of to a current driving condition associatedwith operation of the vehicle.

In some embodiments, the situational awareness may include aphysiological state and/or an inferred emotional state of the person.For example, an emotional state may be estimated or inferred by usingone or more measurements (such as the one or more measurements) asinputs to a pretrained neural network or a machine-learning model thatoutputs an estimate of an emotional state (such as a probability of aparticular emotional state or a classification in a particular emotionalstate).

Moreover, the physiological state may include: an awake state, an alertstate, and/or an oriented state. Alternatively or additionally, thephysiological state may include a vital sign of the person. Note thatthe physiological state may include a change in the vital signcorresponding to a change in a driving condition associated withoperation of the vehicle.

Furthermore, the electronic device may receive a request (operation 814)or a message associated with a partial or fully autonomous vehicleapplication to transition to manual control of the vehicle. For example,the request may correspond to an occurrence of an unknown or an unsafedriving condition associated with operation of the vehicle, which maylead the partial or fully autonomous vehicle application to want totransfer control to the person.

In response to the request, the electronic device may selectivelyprovide the recommendation (operation 816) to the partial or fullyautonomous vehicle application to transition to manual control of thevehicle based at least in part on the situational awareness.

While the preceding example illustrated bilateral communication betweenthe vehicle and the electronic device, in order to facilitate a fasteror real-time response when deciding whether to transition control to theperson, in some embodiments the electronic device may continuously,periodically (such as after a time interval) or as needed (such as whenthere is a change in the assessed situational awareness) update thevehicle as to a current assessment of the situational awareness of theperson. For example, the electronic device may update information storedin memory or in a register (such as a numerical value corresponding tothe assessed situational awareness or a bit indicating whether or notthe person is sufficiently situationally aware). In these embodiments,the vehicle may access this stored information and, thus, may use theinformation when deciding whether or not to transition to manualcontrol. Therefore, in some embodiments, instead of operations 814 and816, the electronic device may provide information that specifies orindicates the assessed situational awareness to the vehicle or may storethis information in memory, which can be accessed by the vehicle, asneeded.

In some embodiments, the electronic device may perform one or moreoptional additional operations (operation 818). For example, when therecommendation is not provided, the electronic device may selectivelyprovide a second recommendation to the partial or fully autonomousvehicle application to not transition to manual control of the vehiclebased at least in part on the request and the situational awareness.

Note that the electronic device may be a portable or removableelectronic device, such as a measurement or sensor module that isinstalled in or integrated into the vehicle.

FIG. 9 presents a drawing illustrating an example of communication amongcomponents in electronic device 112 and vehicle 710. Notably, sensor(s)910 in electronic device 112 may acquire one or more measurements 912 inan environment that is external to electronic device 112, where the oneor more measurements 912 provide information 914 associated with aperson located in a driver's position in vehicle 710, and the one ormore measurements 912 include a non-contact measurement.

Then, an integrated circuit (I.C.) 916 in electronic device 112 mayextract information 914 from the one or more measurements 912, and mayassess situational awareness (S.A.) 920 of the person based at least inpart on the one or more measurements 912 and/or information 914. Forexample, situational awareness 920 may include a physiological stateand/or an inferred emotional state of the person. Notably, thephysiological state may include a change in the vital sign correspondingto a change in a driving condition associated with operation of vehicle710. Thus, in some embodiments, integrated circuit 916 may determine avital sign 918 of the person based at least in part on the one or moremeasurements 912.

Moreover, in response to an occurrence of an unknown or an unsafedriving condition associated with operation of vehicle 710, a partial orfully autonomous vehicle application executed in an environment ofvehicle 710 (such as in an operating system environment) may provide arequest 922 to electronic device 112 to transition to manual control ofvehicle 710 (i.e., to have the person drive vehicle 710). Afterreceiving request 922, a communication circuit 924 in electronic device112 may provide request 922 to integrated circuit 916. Then, in responseto request 922, integrated circuit 916 may, via communication circuit924, selectively provide recommendation 926 to the partial or fullyautonomous vehicle application to transition to manual control ofvehicle 710 based at least in part on situational awareness 920.

FIG. 10 presents a drawing illustrating an example of selectiveproviding a recommendation using electronic device 112 (FIGS. 1 and 2).Notably, sensor(s) may continuously, periodically or as needed, acquireone or more measurements in an environment that is external toelectronic device 112 (such as inside of a vehicle), where the one ormore measurement provides information associated with a person locatedin a driver's position in the vehicle, and the one or more measurementsincludes a non-contact measurement. For example, the one or moremeasurements may include one or more instances radar measurements 1010and one or more instances optical measurement (such as capturing images1012), such as a sequence of measurements at timestamps 1014. Then,electronic device 112 may analyze the radar measurements 1010 to extracta signature 1016 associated with the person and/or to determine a vitalsign (such as a pulse and/or a respiration rate). Moreover, electronicdevice 112 may analyze image 1012 to determine a physiological stateand/or an inferred emotional state of the person. In general, theinformation provided by the one or more measurements may be compared orjointly analyzed.

Next, electronic device 112 may use the one or more measurements toassess situational awareness of a potential driver. The situationalawareness may include a physiological state and/or an inferred emotionalstate of the person. For example, the physiological state may include:an awake state, an alert state, and/or an oriented state. Thus, if apotential driver is drowsy, sleeping, distracted, drunk, panicked (e.g.,overwhelmed by fear) and/or angry, they are not capable of safelyassuming control of a vehicle.

Alternatively or additionally, the physiological state may include avital sign of the person. For example, if the person is situationallyaware and they perceive that there is a risk of an accident based on thecurrent driving condition(s) associated with operation of the vehicle,they may be afraid. This fear is appropriate to the circumstances (i.e.,they should be afraid) and may result in an associated physiologicalresponse, such as a sudden increase in the heart or pulse rate and/orrespiration (e.g., an increase of 5%, 10%, 20% or more with a 3-10seconds). Therefore, the physiological state may include a change in thevital sign corresponding to a change in a driving condition associatedwith operation of the vehicle. When this change is detected, electronicdevice 112 may conclude that the person is situationally aware.

Thus, if there is a potentially dangerous situation, which is leads thepartial or fully autonomous vehicle application to provide the request,and the person reacts in a manner indicative of fear or alarm (but notpanic, so that the person is capable of responding to the currentdriving circumstances), they may be situationally aware and capable ofassuming effective manual control. In this way, the feedback techniquesmay help ensure that a transition to manual control of the vehicleoccurs when it is likely to be positive or constructive as a safetymeasure or failsafe. Consequently, the feedback techniques may helpreduce an accident rate and/or improve a safety performance ofself-driving technology.

While the preceding example illustrated the use of the feedbacktechniques in an automobile, in other embodiments the feedbacktechniques may be used in a wide variety of vehicles that use a partialor fully autonomous vehicle application (e.g., an autopilot), such as anairplane, a ship, etc.

In other embodiments of the feedback techniques, electronic device 112may be used within a vehicle to facilitate automation of environmentalcontrol. Notably, instead of attempting to regulate an environmentalcondition according to a predefined setpoint (such as maintaining adesired temperature), electronic device 112 may be used to automaticallyadjust an environmental condition based at least in part on theperceptions of one or more persons in the environment.

FIG. 11 presents a drawing illustrating an example of an environmentwithin a vehicle 1110. Notably, there may be one or more persons 1112 atdifferent portions or regions 1114 in the environment. Moreover, theremay be one or more barriers at a periphery of the environment, such aswindows 1116. An environmental control system (E.C.S.) 1118 in vehicle1110 may maintain one or more environmental conditions in theenvironment, such as the temperature, relative humidity, and/orvisibility through one or more of windows 1116. Embodiments of thefeedback techniques may be used to assist environmental control system1118, such as by automating modifications to one or more environmentalcontrols or one or more environmental control settings.

This is shown in FIG. 12, which presents a flow diagram illustrating anexample of a method 1200 for automatically modifying an environmentalcontrol. This method may be performed by an electronic device (such aselectronic device 112 in FIGS. 1 and 2) or a component in the electronicdevice (such as an integrated circuit or a processor). During operation,the electronic device may acquire, using one or more sensors, ameasurement (operation 1210) within an environment that is external tothe electronic device, where the measurement includes a non-contactmeasurement associated with a window or one or more biological lifeformsin the environment. Note that the one or more sensors may include: atransmitter that transmits wireless signals and a receiver that receiveswireless-return signals; and/or an image sensor. Alternatively oradditionally, the one or more sensors may include an image sensor thatperforms optical measurements (such as capturing one or more images or avideo).

Then, the electronic device may determine an environmental condition(operation 1212) based at least in part on the measurement.

Next, the electronic device may automatically modify the environmentalcontrol (operation 1214) associated with the environment based at leastin part on the environmental condition. Note that modifying theenvironmental control may include changing one or more of: a thermostatsetting (such as a temperature setpoint), a fan setting (such as on oroff, a blower speed, etc.), a fan direction (such as directing an airflow onto a window), a state of a seat heater (such as on or off, anamount of heating, etc.), a state of seat cooling (such as on or off, anamount of seat cooling, etc.), a state of air conditioning (such as onor off, an amount of air conditioning, etc.), a windshield wiper state(such as on or off, a windshield wiper speed, etc., e.g., when theenvironmental condition indicates that the biological lifeform is havingdifficulty seeing through the window), a state of a defrost or defoggingcircuit (such as on or off), etc.

For example, the environmental condition may include presence of ice ona first surface of the window, and the environmental control mayactivate de-icing of the first surface of the window. Alternatively oradditionally, the environmental condition may include presence ofcondensed water vapor on a second surface of the window, and theenvironmental control may activate air conditioning and may increaseand/or direct airflow onto the second surface or may activate a defrostor defogging circuit.

Furthermore, the environmental condition may include perception oftemperature in the environment by a biological lifeform in theenvironment, and the environment control may adapt a temperature in atleast a portion of the environment that includes the biologicallifeform. In some embodiments, the electronic device may determine orestimate the perception based at least in part on: a behavior of thebiological lifeform and/or an inferred emotional state of the biologicallifeform.

Additionally, the environmental condition may include differentperceptions of temperature in the environment by a first biologicallifeform and a second biological lifeform in the environment, theenvironment control may adapt a first temperature in a first portion ofthe environment that include the first biological lifeform and may adapta second temperature in a second portion of the environment that includethe second biological lifeform, and the first temperature may bedifferent from the second temperature. In some embodiments, adapting agiven temperature may include: changing one or more of: a thermostatsetting, a fan setting, a fan direction, a state of a seat heater, astate of seat cooling, a state of air conditioning, a windshield wiperstate, a state of a defrost or defogging circuit, etc.

Note that the environmental condition may include detecting a presenceof a biological lifeform in a region in the environment. Thus, theenvironmental control may be automatically modified when a person isdetected in, e.g., a passenger seat in a car.

In some embodiments, the electronic device performs one or more optionaladditional operations (operation 1216).

Note that the electronic device may be a portable or removableelectronic device, such as a measurement or sensor module that isinstalled in or integrated into the vehicle.

In some embodiments of method 400 (400), 800 (FIG. 8) and/or 1200 theremay be additional or fewer operations. Moreover, the order of theoperations may be changed, and/or two or more operations may be combinedinto a single operation.

FIG. 13 presents a drawing illustrating an example of communicationamong components in electronic device 112 and vehicle 1110. Notably,sensor(s) 1310 in electronic device 112 may acquire one or moremeasurements 1312 in an environment that is external to electronicdevice 112, where the one or more measurements 1312 provide information1314 associated with a window or one or more biological lifeforms (suchas one or more people or one or more animals) in the environment, andthe one or more measurements 1312 include a non-contact measurement.

Then, an integrated circuit (I.C.) 1316 in electronic device 112 mayextract information 1314 based at least in part on the one or moremeasurements 1312, and may determine an environmental condition 1324based at least in part on the one or more measurements 1312 and/orinformation 1314. For example, environmental condition 1324 may includeperception of temperature in the environment by a biological lifeform inthe environment (such as that the temperature is too cold or feels toocold). In some embodiments, integrated circuit 1316 may determine orestimate the perception based at least in part on: a behavior 1318 ofthe biological lifeform (such as shivering, folding or wrapping theirarms around their torso, a sound made by a person, e.g., ‘brrr’, statingthat they are warm or cold, non-verbal communication, a facialexpression, etc.), an inferred emotional state 1320 of the biologicallifeform and/or a vital sign 1322 of the biological lifeform.

Next, integrated circuit 1316 may automatically modify environmentalcontrol 1326 associated with the environment based at least in part onenvironmental condition 1324. For example, integrated circuit 1316 mayprovide the modified environmental control 1326 to communication circuit1328 in electronic device 112, which provides the modified environmentalcontrol 1326 to vehicle 1110. In response, vehicle 1110 may change oneor more of: a thermostat setting, a fan setting, a fan direction, astate of a seat heater, a state of seat cooling, a state of airconditioning, a windshield wiper state, a defrost or defogging circuit,etc. Thus, in response to a biological lifeform perceiving that thetemperature in the environment is too cold or feels too cold, themodified environment control 1326 may adapt a temperature in at least aportion of the environment in vehicle 1110 that includes the biologicallifeform.

While communication between the components in FIGS. 5, 9 and/or 13 isillustrated with unilateral or bilateral communication (e.g., lineshaving a single arrow or dual arrows), in general a given communicationoperation may be unilateral or bilateral.

FIG. 14 presents a drawing illustrating an example of automaticallymodifying an environmental control using electronic device 112 (FIGS. 1and 2). Notably, sensor(s) may continuously, periodically or as needed,acquire one or more measurements in an environment that is external toelectronic device 112 (such as inside of a vehicle), where the one ormore measurements include a non-contact measurement associated with awindow or one or more biological lifeforms in the environment (such asone or more people and/or an animal, e.g., a pet). For example, the oneor more measurements may include radar measurements 1410 and/or anoptical measurement (such as capturing an image 1412) at differenttimestamps 1414.

Then, electronic device 112 may analyze the radar measurements 1410 toextract a signature 1416 associated with the window or the one or morebiological lifeforms, and then may determine a classification or anoccurrence of an environmental condition based at least in part onsignature 1416. Alternatively or additionally, electronic device mayanalyze the image to determine the classification or the occurrence ofthe environmental condition based at least in part on the informationincluded in image 1412. For example, electronic device 112 may detectthe presence of ice, fog, moisture or condensation (such as condensedwater vapor) on an interior or an exterior surface of the window.

Furthermore, the environmental condition may include perception oftemperature in the environment by a person in the environment. Notably,the perception may be determined or estimated based at least in part on:a behavior of the person and/or an inferred emotional state of theperson. For example, the behavior may include shivering, folded arms,verbal complaining, a facial expression, etc., that indicates that theperson feels cold (regardless of the actual temperature around theperson in one of regions 1114 in FIG. 11). Alternatively oradditionally, the behavior may include sweating (such as the presence ofsweat on a person's skin or their clothing may be soaked), the personmay be fanning themselves, verbal complaining, a facial expression,etc., that indicates that the person feels hot (regardless of the actualtemperature around the person in one of regions 1114 in FIG. 11). Insome embodiments, the behavior may include squinting or wiping awaymoisture or condensed water vapor from a portion of a window, which mayindicate that the person is having difficulty seeing through the window.Additionally, the emotional state may include irritation. In these andother embodiments in the present disclosure, an emotional state may beestimated by using the one or more measurements as inputs to apretrained neural network or a machine-learning model that outputs anestimate of an emotional state (such as a probability of a particularemotional state or a classification in a particular emotional state).

Based on the determined environmental condition, electronic device 112may automatically modify the environmental control or may provide aninstruction to an environmental control system. When the environmentalcondition involves perception of temperature, this modification mayadjust a temperature in at least one of regions 1114 (FIG. 11) bychanging one or more of: a temperature setpoint, a fan setting, a stateof a seat heater, a state of seat heating, a state of air conditioning,etc. Alternatively, when the environmental condition involves acondition related to a window, the modification may adjust one or moreof: a windshield wiper state, a state of a defrost or defogging circuit,a state of air conditioning, a fan setting, a fan direction, etc. Notethat in the present discussion a ‘state’ may include: on, off, a realvalue corresponding to an amount between a minimum or a maximum amount,etc.

While the preceding example illustrated the use of the feedbacktechniques in an automobile, in other embodiments the feedbacktechniques may be used in a wide variety of vehicles or environmentsthat include an environmental control system, such as an airplane, aship, an office, a room, a building, etc.

We now describe embodiments of an electronic device, which may performat least some of the operations in the security and/or the feedbacktechniques. FIG. 15 presents a block diagram illustrating an example ofan electronic device 1500, such as electronic device 112 (FIGS. 1 and2). This electronic device may include processing subsystem 1510, memorysubsystem 1512, networking subsystem 1514 and sensor subsystem 1530.Processing subsystem 1510 includes one or more devices configured toperform computational operations. For example, processing subsystem 1510can include one or more microprocessors, ASICs, microcontrollers,programmable-logic devices, graphical processor units (GPUs) and/or oneor more digital signal processors (DSPs).

Memory subsystem 1512 includes one or more devices for storing dataand/or instructions for processing subsystem 1510 and networkingsubsystem 1514. For example, memory subsystem 1512 can include dynamicrandom access memory (DRAM), static random access memory (SRAM), and/orother types of memory (which collectively or individually are sometimesreferred to as a ‘computer-readable storage medium’). In someembodiments, instructions for processing subsystem 1510 in memorysubsystem 1512 include: one or more program modules or sets ofinstructions (such as program instructions 1522 or operating system1524), which may be executed by processing subsystem 1510. Note that theone or more computer programs may constitute a computer-programmechanism. Moreover, instructions in the various modules in memorysubsystem 1512 may be implemented in: a high-level procedural language,an object-oriented programming language, and/or in an assembly ormachine language. Furthermore, the programming language may be compiledor interpreted, e.g., configurable or configured (which may be usedinterchangeably in this discussion), to be executed by processingsubsystem 1510.

In addition, memory subsystem 1512 can include mechanisms forcontrolling access to the memory. In some embodiments, memory subsystem1512 includes a memory hierarchy that comprises one or more cachescoupled to memory in electronic device 1500. In some of theseembodiments, one or more of the caches is located in processingsubsystem 1510.

In some embodiments, memory subsystem 1512 is coupled to one or morehigh-capacity mass-storage devices (not shown). For example, memorysubsystem 1512 can be coupled to a magnetic or optical drive, asolid-state drive, or another type of mass-storage device. In theseembodiments, memory subsystem 1512 can be used by electronic device 1500as fast-access storage for often-used data, while the mass-storagedevice is used to store less frequently used data.

Networking subsystem 1514 includes one or more devices configured tocouple to and communicate on a wired and/or wireless network (i.e., toperform network operations), including: control logic 1516, an interfacecircuit 1518 and one or more antennas 1520 (or antenna elements). (WhileFIG. 15 includes one or more antennas 1520, in some embodimentselectronic device 1500 includes one or more nodes, such as nodes 1508,e.g., a pad, which can be coupled to the one or more antennas 1520.Thus, electronic device 1500 may or may not include the one or moreantennas 1520.) For example, networking subsystem 1514 can include aBluetooth networking system, a cellular networking system (e.g., a3G/4G/5G network such as UMTS, LTE, etc.), a USB networking system, anetworking system based on the standards described in IEEE 802.11 (e.g.,a Wi-Fi networking system), an Ethernet networking system, and/oranother networking system.

Note that a transmit or receive antenna pattern (or antenna radiationpattern) of electronic device 1500 may be adapted or changed usingpattern shapers (such as reflectors) in one or more antennas 1520 (orantenna elements), which can be independently and selectivelyelectrically coupled to ground to steer the transmit antenna pattern indifferent directions. (Alternatively or additionally, the transmit orreceive antenna pattern may be adapted or changed using a phased array.)Thus, if one or more antennas 1520 include N antenna pattern shapers,the one or more antennas may have 2^(N) different antenna patternconfigurations. More generally, a given antenna pattern may includeamplitudes and/or phases of signals that specify a direction of the mainor primary lobe of the given antenna pattern, as well as so-called‘exclusion regions’ or ‘exclusion zones’ (which are sometimes referredto as ‘notches’ or ‘nulls’). Note that an exclusion zone of the givenantenna pattern includes a low-intensity region of the given antennapattern. While the intensity is not necessarily zero in the exclusionzone, it may be below a threshold, such as 3 dB or lower than the peakgain of the given antenna pattern. Thus, the given antenna pattern mayinclude a local maximum (e.g., a primary beam) that directs gain in thedirection of electronic device 1500 that is of interest, and one or morelocal minima that reduce gain in the direction of other electronicdevices that are not of interest. In this way, the given antenna patternmay be selected, e.g., to target an object of interest in an environmentof electronic device 1500.

Networking subsystem 1514 includes processors, controllers,radios/antennas, sockets/plugs, and/or other devices used for couplingto, communicating on, and handling data and events for each supportednetworking system. Note that mechanisms used for coupling to,communicating on, and handling data and events on the network for eachnetwork system are sometimes collectively referred to as a ‘networkinterface’ for the network system. Moreover, in some embodiments a‘network’ or a ‘connection’ between the electronic devices does not yetexist. Therefore, electronic device 1500 may use the mechanisms innetworking subsystem 1514 for performing simple wireless communicationbetween the electronic devices, e.g., transmitting frames and/orscanning for frames transmitted by other electronic devices.

Within electronic device 1500, processing subsystem 1510, memorysubsystem 1512, and networking subsystem 1514 are coupled together usingbus 1528. Bus 1528 may include an electrical, optical, and/orelectro-optical connection that the subsystems can use to communicatecommands and data among one another. Although only one bus 1528 is shownfor clarity, different embodiments can include a different number orconfiguration of electrical, optical, and/or electro-optical connectionsamong the subsystems.

In some embodiments, electronic device 1500 includes an optional displaysubsystem 1526 for displaying information on a display, which mayinclude a display driver and the display, such as a liquid-crystaldisplay, a multi-touch touchscreen, etc.

Furthermore, electronic device 1500 may include a sensor subsystem 1530,which may include one or more radar sensors 1532 with one or moretransmitters, one or more receivers, one or more sets of transmitantennas and one or more sets of receive antennas that perform MIMOradar measurements. In some embodiments, sensor subsystem 1530 includesone or more image sensors that acquire images (such as a CCD or a CMOSsensor) and/or one or more additional sensors 1534 (such as alight-intensity sensor, radar, sonar, lidar, etc.). These other oradditional sensors may be used separately or in conjunction with the oneor more radar sensors 1532.

Electronic device 1500 can be (or can be included in) a wide variety ofelectronic devices. For example, electronic device 1500 can be (or canbe included in): a desktop computer, a laptop computer, asubnotebook/netbook, a server, a computer, a mainframe computer, acloud-based computer, a tablet computer, a smartphone, a cellulartelephone, a smartwatch, a consumer-electronic device, a portablecomputing device, a transceiver, a measurement device, anotherelectronic device and/or a vehicle.

Although specific components are used to describe electronic device1500, in alternative embodiments, different components and/or subsystemsmay be present in electronic device 1500. For example, electronic device1500 may include one or more additional processing subsystems, memorysubsystems, networking subsystems, di splay subsystems and/or sensorsubsystems. Additionally, one or more of the subsystems may not bepresent in electronic device 1500. Moreover, in some embodiments,electronic device 1500 may include one or more additional subsystemsthat are not shown in FIG. 15. Also, although separate subsystems areshown in FIG. 15, in some embodiments some or all of a given subsystemor component can be integrated into one or more of the other subsystemsor component(s) in electronic device 1500. For example, in someembodiments program instructions 1522 are included in operating system1524 and/or control logic 1516 is included in interface circuit 1518.

Moreover, the circuits and components in electronic device 1500 may beimplemented using any combination of analog and/or digital circuitry,including: bipolar, PMOS and/or NMOS gates or transistors. Furthermore,signals in these embodiments may include digital signals that haveapproximately discrete values and/or analog signals that have continuousvalues. Additionally, components and circuits may be single-ended ordifferential, and power supplies may be unipolar or bipolar.

An integrated circuit (which is sometimes referred to as a‘communication circuit’ or a ‘means for communication’) may implementsome or all of the functionality of networking subsystem 1514 or sensorsubsystem 1530. The integrated circuit may include hardware and/orsoftware mechanisms that are used for transmitting wireless or radarsignals from electronic device 1500 and receiving wireless or radarsignals at electronic device 1500 from other electronic devices. Asidefrom the mechanisms herein described, radios are generally known in theart and hence are not described in detail. In general, networkingsubsystem 1514 and/or the integrated circuit can include any number ofradios. Note that the radios in multiple-radio embodiments function in asimilar way to the described single-radio embodiments.

In some embodiments, networking subsystem 1514 and/or the integratedcircuit include a configuration mechanism (such as one or more hardwareand/or software mechanisms) that configures the radio(s) to transmitand/or receive on a given communication channel (e.g., a given carrierfrequency). For example, in some embodiments, the configurationmechanism can be used to switch the radio from monitoring and/ortransmitting on a given communication channel to monitoring and/ortransmitting on a different communication channel. (Note that‘monitoring’ as used herein comprises receiving signals from otherelectronic devices and possibly performing one or more processingoperations on the received signals)

Moreover, another integrated circuit may implement some or all of thefunctionality related to the security and/or the feedback techniques.

In some embodiments, an output of a process for designing a givenintegrated circuit, or a portion of the given integrated circuit, whichincludes one or more of the circuits described herein may be acomputer-readable medium such as, for example, a magnetic tape or anoptical or magnetic disk. The computer-readable medium may be encodedwith data structures or other information describing circuitry that maybe physically instantiated as the given integrated circuit or theportion of the given integrated circuit. Although various formats may beused for such encoding, these data structures are commonly written in:Caltech Intermediate Format (CIF), Calma GDS II Stream Format (GDSII) orElectronic Design Interchange Format (EDIF). Those of skill in the artof integrated circuit design can develop such data structures fromschematics of the type detailed above and the corresponding descriptionsand encode the data structures on the computer-readable medium. Those ofskill in the art of integrated circuit fabrication can use such encodeddata to fabricate integrated circuits that include one or more of thecircuits described herein.

While some of the operations in the preceding embodiments wereimplemented in hardware or software, in general the operations in thepreceding embodiments can be implemented in a wide variety ofconfigurations and architectures. Therefore, some or all of theoperations in the preceding embodiments may be performed in hardware, insoftware or both. For example, at least some of the operations in thesecurity or the feedback techniques may be implemented using programinstructions 1522, operating system 1524 (such as a driver for interfacecircuit 1518) or in firmware in interface circuit 1518. Alternatively oradditionally, at least some of the operations in the security or thefeedback techniques may be implemented in a physical layer, such ashardware in interface circuit 1518 or sensor subsystem 1530.

While the preceding embodiments illustrated the use of a vehicle, suchas a car, a truck, a bus, etc., in other embodiments the security and/orthe feedback techniques are used in conjunction with a flying vehicle(such as a drone, a helicopter, an airplane, etc.), a boat or a ship,and/or a submersible vehicle (such as a drone or a submarine).

In the preceding description, we refer to ‘some embodiments.’ Note that‘some embodiments’ describes a subset of all of the possibleembodiments, but does not always specify the same subset of embodiments.Note that numerical values in the preceding embodiments are illustrativeexamples of some embodiments. In other embodiments of the security orthe feedback techniques, different numerical values may be used.

The foregoing description is intended to enable any person skilled inthe art to make and use the disclosure, and is provided in the contextof a particular application and its requirements. Moreover, theforegoing descriptions of embodiments of the present disclosure havebeen presented for purposes of illustration and description only. Theyare not intended to be exhaustive or to limit the present disclosure tothe forms disclosed. Accordingly, many modifications and variations willbe apparent to practitioners skilled in the art, and the generalprinciples defined herein may be applied to other embodiments andapplications without departing from the spirit and scope of the presentdisclosure. Additionally, the discussion of the preceding embodiments isnot intended to limit the present disclosure. Thus, the presentdisclosure is not intended to be limited to the embodiments shown, butis to be accorded the widest scope consistent with the principles andfeatures disclosed herein.

What is claimed is:
 1. An electronic device, comprising: one or moresensors, wherein the one or more sensors comprise one or more of: aradar sensor, a first optical sensor configured to acquire informationin a visible band of frequencies, or a second optical sensor configuredto acquire information in an infrared band of frequencies; an integratedcircuit, coupled to the one or more sensors, configured to: acquire,using the one or more sensors, a measurement in an environment that isexternal to the electronic device, wherein the measurement providesinformation associated with an object, and wherein the measurementcomprises a non-contact measurement; detect the object based at least inpart on the measurement; estimate an intent of the object based at leastin part on the measurement; and when the estimated intent is associatedwith a type of adverse event, perform a preventive action prior to anoccurrence of the type of adverse event, wherein the preventive actionreduces a probability of the occurrence of the type of adverse event oran amount of financial damage associated with the occurrence of the typeof adverse event.
 2. The electronic device of claim 1, wherein the oneor more sensors comprises a transmitter configured to transmit wirelesssignals and a receiver configured to receive wireless-return signals. 3.The electronic device of claim 1, wherein the one or more sensorscomprise two or more different types of sensors and the measurement isacquired using the two or more different types of sensors.
 4. Theelectronic device of claim 1, wherein the object comprises a person andthe integrated circuit is configured to identify the person; and whereinthe intent is estimated based at least in part on the identity of theperson.
 5. The electronic device of claim 1, wherein the objectcomprises a person having an unknown identity; and wherein the intent isestimated based at least in part on an association of the person withone or more prior occurrences of the type of adverse event in an eventhistory.
 6. The electronic device of claim 1, wherein the type ofadverse event comprises: theft, property damage, or another type ofcrime.
 7. The electronic device of claim 1, wherein the measurement isacquired using a first sensor in the one or more sensors; and whereinthe preventive action comprises acquiring, using a second sensor in theone or more sensors, a second measurement that provides informationassociated with the object.
 8. The electronic device of claim 1, whereinthe electronic device comprises a light source and the integratedcircuit is coupled to the light source; and wherein the preventiveaction comprises selectively illuminating the object using the lightsource.
 9. The electronic device of claim 1, wherein the electronicdevice comprises an alarm and the integrated circuit is coupled to thealarm; and wherein the preventive action comprises selectivelyactivating the alarm.
 10. The electronic device of claim 1, wherein thepreventive action comprises providing information about the objectaddressed to a law enforcement agency or contacting a law enforcementagency.
 11. The electronic device of claim 1, wherein the electronicdevice comprises a lock and the integrated circuit is coupled to thelock; and wherein the preventive action comprises: determining a stateof the lock; when the lock is in an unlocked state, providing anelectronic signal that sets the lock into a locked state; and disablingan ability to change the state of the lock.
 12. The electronic device ofclaim 1, wherein the electronic device comprises a vehicle; and whereinthe preventive action comprises disabling movement of the vehicle. 13.The electronic device of claim 1, wherein the electronic devicecomprises a battery coupled to the integrated circuit; and wherein theintegrated circuit is configured to perform at least one of theacquiring, the detecting, the estimating and the performing using adynamic subset of resources in the electronic device based at least inpart on a discharge current of the battery, a remaining charge of thebattery, or both.
 14. The electronic device of claim 1, wherein theobject comprises a person and the intent is estimated based at least inpart on an inferred emotional state of the person or a behavior of theperson.
 15. The electronic device of claim 1, wherein the measurementprovides information associated with a second object that is related tothe object, the integrated circuit is configured to identify the secondobject, and the intent is estimated based at least in part on theidentified second object.
 16. The electronic device of claim 1, whereinthe object comprises a person and the estimated intent is based at leastin part on a vital sign of the person, which is specified by themeasurement.
 17. A vehicle, comprising: one or more sensors, wherein theone or more sensors comprise one or more of: a radar sensor, a firstoptical sensor configured to acquire information in a visible band offrequencies, or a second optical sensor configured to acquireinformation in an infrared band of frequencies; an integrated circuit,coupled to the one or more sensors, configured to: acquire, using theone or more sensors, a measurement in an environment that is external tothe vehicle, wherein the measurement provides information associatedwith an object, and wherein the measurement comprises a non-contactmeasurement; detect the object based at least in part on themeasurement; estimate an intent of the object based at least in part onthe measurement; and when the estimated intent is associated with a typeof adverse event, perform a preventive action prior to an occurrence ofthe type of adverse event, wherein the preventive action reduces aprobability of the occurrence of the type of adverse event or an amountof financial damage associated with the occurrence of the type ofadverse event.
 18. The vehicle of claim 17, wherein the object comprisesa person and the intent is estimated based at least in part on aninferred emotional state of the person or a behavior of the person. 19.A non-transitory computer-readable storage medium for use in conjunctionwith an electronic device, the computer-readable storage medium storingprogram instructions, wherein, when executed by the computer system, theprogram instructions cause the computer system to perform one or moreoperations comprising: acquiring, using one or more sensors, ameasurement in an environment that is external to the electronic device,wherein the one or more sensors comprise one or more of: a radar sensor,a first optical sensor configured to acquire information in a visibleband of frequencies, or a second optical sensor configured to acquireinformation in an infrared band of frequencies, wherein the measurementprovides information associated with an object, and wherein themeasurement comprises a non-contact measurement; detecting the objectbased at least in part on the measurement; estimating an intent of theobject based at least in part on the measurement; and when the estimatedintent is associated with a type of adverse event, performing apreventive action prior to an occurrence of the type of adverse event,wherein the preventive action reduces a probability of the occurrence ofthe type of adverse event or an amount of financial damage associatedwith the occurrence of the type of adverse event.
 20. A method forperforming a preventive action by an electronic device, comprising:acquiring, using one or more sensors, a measurement in an environmentthat is external to the electronic device, wherein the one or moresensors comprise one or more of: a radar sensor, a first optical sensorconfigured to acquire information in a visible band of frequencies, or asecond optical sensor configured to acquire information in an infraredband of frequencies, wherein the measurement provides informationassociated with an object, and wherein the measurement comprises anon-contact measurement; detecting the object based at least in part onthe measurement; estimating an intent of the object based at least inpart on the measurement; and when the estimated intent is associatedwith a type of adverse event, performing the preventive action prior toan occurrence of the type of adverse event, wherein the preventiveaction reduces a probability of the occurrence of the type of adverseevent or an amount of financial damage associated with the occurrence ofthe type of adverse event.